The following definitions will be helpful in explaining the known background elements that are helpful for understanding the present invention. The “personal computer” or simply “PC” is a term that is so often used it seems unlikely, at first, to require formal definition. However the precise scope of the term is sometimes vague. A PC is a computer whose size, and capabilities (and some have said price) make it useful for individuals, intended to be operated directly by an end user and capable of performing a variety of general purpose tasks, with no intervening computer operator. The PC may be a home computer, or may be found in an office, a medical facility or a research lab. The PC may often be connected to a local area network. The distinguishing characteristics of a PC are that the computer is primarily used, interactively, by one person at a time. This is opposite to the batch processing or time-sharing models which allowed large expensive systems to be used by many people, usually at the same time, or large data processing systems which required a full-time staff to operate. The PC can come in desktop models, notebook models, handheld models, and hybrids of these.
A “notebook computer”, or simply “notebook” within this application, is an extremely lightweight PC. Notebook computers typically weigh less than 6 pounds and are small enough to fit easily in a briefcase. Aside from size and portability, the principal difference between a notebook computer and a non-notebook personal computer (e.g. a desktop computer) is the display screen. Notebook computers use a variety of techniques, known as flat-pane technologies, to produce a lightweight and non-bulky display screen. Laptop computers and tablet PCs are two types of notebook computers. Usually all of the interface hardware needed to operate the notebook computer, such as parallel and serial ports, graphics card, sound channel, etc., are built in to a single unit. Most notebook computers contain batteries to facilitate operation without a readily available electrical outlet.
A “laptop computer”, or simply laptop, is, within this application, a subset of notebooks. A laptop will have a display and separate keyboard interface (e.g. “qwerty” keyboard), with the keyboard and the display typically hinged together. The term Laptop is sometimes used more broadly and equated with notebooks, but the term will have a narrower definition within this application.
A “Tablet PC” or is a notebook computer, also called a panel computer, and was first introduced by Pen Computing in the early 90s with their PenGo Tablet Computer and popularized by Microsoft. Its touch-screen or “graphics tablet/screen hybrid technology” technology allows the user to operate the computer with a stylus or digital pen, or a fingertip, instead of a keyboard or mouse. A tablet computer, or simply tablet, is a one-piece mobile computer. Tablets typically offer a touch screen, with finger (or stylus) gestures acting as the primary means of control, though often supplemented by the use of one or more physical context sensitive buttons or the input from one or more accelerometers; an on-screen, hideable virtual keyboard is generally offered as the principle means of data input. The term tablet as used herein will encompass personal computing devices also commonly called smart phones and the more antiquated personal digital assistants.
Though generally self contained a tablet computer may be connected to a physical keyboard (or other input device), and a number of hybrids that offer a detachable keyboard have been marketed since the mid 1990's, as have a number of convertible touch screen notebook computers that offer an integrated keyboard that can be hidden by a swivel joint or slide joint, exposing only the screen for touch operation. Tablets have also appeared in a foldable Booklet format that offer the user dual-touchscreens, and can be used as a notebook by displaying a virtual keyboard on one of display.
The input/output ports of a personal computer refer to the communications links through which the personal computers send and receive information, which generally include serial ports, parallel ports, wireless links or connectors (such as WI-FI and Bluetooth), and universal serial bus (USB) ports. In addition, some laptops have expansion slots for PCMCIA standard adaptor cards (Type I and Type II) that also form input/output ports.
A personal digital assistant (PDA), also known as a palmtop computer, or personal data assistant, is a defined herein as a type of tablet and is a mobile device that functions as a personal information manager. PDAs are generally considered obsolete with the widespread adoption of smart phones. Nearly all current PDAs have the ability to connect to the Internet. A PDA has an electronic visual display, enabling it to include a web browser, all current models also have audio capabilities enabling use as a portable media player, and also enabling most of them to be used as mobile phones. Most PDAs can access the Internet, intranets or extranets via Wi-Fi or Wireless Wide Area Networks. Most PDAs employ touch screen technology.
A smart phone is a mobile phone built on a mobile operating system, with more advanced computing capability connectivity than a feature phone, and thus is included herein under the broader definition of tablet computer. The first smart phones combined the functions of a personal digital assistant (PDA) with a mobile phone. More recent models added the functionality of portable media players, low-end compact digital cameras, pocket video cameras, and GPS navigation units to form one multi-use device. Many modern smart phones also include high-resolution touch screens and web browsers that display standard web pages as well as mobile-optimized sites. High-speed data access is provided by Wi-Fi and mobile broadband. In recent years, the rapid development of the mobile application, or mobile apps, market and of mobile commerce have been drivers of smart phone adoption.
The mobile operating systems (OS) used by modern smart phones include Google's Android, Apple's iOS, Nokia's Symbian, RIM's BlackBerry OS, Samsung's Bada, Microsoft's Windows Phone, Hewlett-Packard's webOS, and embedded Linux distributions such as Maemo and MeeGo. Such operating systems can be installed on many different phone models, and typically each device can receive multiple OS software updates over its lifetime. A few other representative operating systems are Mozilla's Firefox OS, Canonical Ltd.'s Ubuntu Phone, and Tizen.
As a representative example of an operating system Android is a Linux-based operating system designed primarily for touch screen mobile devices such as smart phones and tablet computers. Initially developed by Android, Inc., which Google backed financially and later purchased in 2005, Android was unveiled in 2007 along with the founding of the Open Handset Alliance: a consortium of hardware, software, and telecommunication companies devoted to advancing open standards for mobile devices. The first Android-powered phone was sold in October 2008.
Android is open source and Google releases the code under the Apache License. This open source code and permissive licensing allows the software to be freely modified and distributed by device manufacturers, wireless carriers and enthusiast developers. Additionally, Android has a large community of developers writing mobile applications that extend the functionality of devices, written primarily in a customized version of the Java programming language. For example, in October 2012, there were approximately 700,000 applications available for Android, and the estimated number of applications downloaded from Google Play, Android's primary app store, was 25 billion.
These factors have allowed Android to become the world's most widely used smartphone platform, overtaking Symbian in the fourth quarter of 2010, and the software of choice for technology companies who require a low-cost, customizable, lightweight operating system for high tech devices without developing one from scratch. Android's open nature has further encouraged a large community of developers and enthusiasts to use the open source code as a foundation for community-driven projects, which add new features for advanced users or bring Android to devices which were officially released running other operating systems. Android had a worldwide smart phone market share of 75% during the third quarter of 2012, with 500 million devices activated in total and 1.3 million activations per day.
The present invention relates to monitoring of physically sensed parameters of a given environment with one or more user selectable sensors in portable manner that allows the user to integrate real time contemporaneous comments in any recorded session.
Physical parameters are measured with any of an almost infinite number of available sensors. The list of physical sensors is extremely large and constantly growing. An illustrative non-exhaustive list of sensors known in the art include temperature sensors, humidity sensors, barometers, gyroscope, positional (e.g., GPS) sensors, pressure sensors, flow rate sensors (e.g. wind speed, water flow rates, etc), magnetism field or flux sensors, electrical field or flux sensors, gas sensors (e.g. carbon monoxide sensors, carbon dioxide sensors or radon gas detectors), particulate counters, air quality meters, discrete chemical sensors, motion sensors, strain gauges, radar sensors, sonar sensors, Lidar sensors, microwave sensors, volatile organic compound sensors, Nox sensors, CxO sensors, volt meters, thermal imaging sensors, accelerometers, video or spectral sensors, radiation sensors, acoustic sensors, seismic sensor, and thermal conductivity sensors.
The above incomplete listing is pulled mainly from sensors utilized in various inspection and portable surveillance fields in which the present mobile application has a large number of practical applications. Every field can yield its own unique set of available sensors, for example the medical field yields a number of physiologic sensors, which within the meaning of this specification is a sensor that measures a parameter related to a physical characteristic of a living subject, such as a human or animal. The types of physiologic sensors include, for example, blood pressure sensors, blood flow sensors, blood glucose sensors, blood cholesterol sensors, heart sound sensors, EMG sensors, EEG sensors, EKG sensors, EOG sensors, pulse sensors, oxygenation sensors, pulse-oximetry sensors, blood perfusion sensors, respiration sensors (both pressure, flow and rate), temperature sensors, additional blood gas sensors (such as nitrogen partial pressure, carbon dioxide partial pressure, carbon monoxide partial pressure, oxygen partial pressure, and pH level), motion sensors, strain gauges, body position sensors, limb motion sensors and the like. The medical field often does not need to utilize the portable aspects of the present invention, except for first responders, for whom integrating sensors with video and responder commenting could be useful for improving patient diagnosis and treatment following initial onsite emergency, transport and triage care. The medical field has been an early technology adopter or innovator accommodating and integrating multiple sensor inputs into a single device.
U.S. Patent Application Publication 2009-0275809 discloses a kiosk based portable modular kiosk based physiologic sensor system for clinical and research applications which is incorporated herein by reference and which teaches the modular implementation of multiple sensors on a tablet computer. A wide variety of medical sensors are disclosed, but the platform does not appear to be universally expandable, nor particularly portable and there is no ability for user controlled commenting and flagging that is necessary for many applications such as real time inspections and surveillance or real time contemporaneous notation.
U.S. Patent Application Publication 2002-0045835 discloses a notebook computer based on noninvasive physiologic sensor platform that is of general interest and is incorporated herein by reference. This reference discloses some aspects of sensor integration in a PC computer environment.
U.S. Published Patent Application 2006-0264762 discloses a personal computer (PC) based physiologic monitor system that includes a personal computer having a display and an input/output port for attachment to an external device. The PC based system also includes a physiologic sensor coupled to the personal computer through the input/output port so that a modified output of the physiologic sensor is graphically displayed on the display. A controller, a portion of which is disposed in the personal computer, modifies the output of the physiologic sensor and provides a feedback control signal for modifying the output of the physiologic sensor. This disclosure is incorporated herein by reference.
U.S. Patent Application Publication 2008-0146277, which is incorporated herein by reference, discloses a personal healthcare assistant that includes a variety of data devices which are connected to a telephonic connection to a healthcare provider via wireless connections to provide a variety of desired data to the healthcare provider and which can be partially controlled by the healthcare provider. Again a wide variety of medical sensors are disclosed, but the platform does not appear to be universally expandable, nor particularly portable and there is no ability for user controlled commenting and flagging that is necessary for many applications such as real time inspections and surveillance or real time contemporaneous notation.
Outside of the medical field, per se, is “Sensor Platforms' FreeMotion Library” which provides for the combination of process data from sensors, such as accelerometers, barometers, gyroscopes and magnetometers, on mobile devices to allow users to “interpret that data into contextual information meaningful to application developers.” The FreeMotion Library works with Android and Windows 8 operating systems, and can also be embedded in microcontrollers and/or smart sensors. The FreeMotion Library utilizes an algorithmic technique of combining the outputs from two or more sensors recording a common event, providing a “fused result.” The particular approach used here may be considered somewhat restrictive to allow for users to freely adopt any desired sensor output as the sensor outputs must fit the “algorithmic technique of combining the outputs”, however this teaching illustrates that the general integration of sensor data is known in the art for use with mobile computing bases. A wide variety of sensors are disclosed, but the platform does not appear to be universally expandable to a variety of sensors, nor particularly portable and there is no ability for user controlled commenting and flagging that is necessary for many applications such as real time inspections and surveillance or real time contemporaneous notation.
U.S. Patent Application Publication 2008-0145830 discloses a method and associated system for computer readable recording medium for synchronous multimedia recording and playback with end user control of time, data and event visualization for playback control over a network. This disclosure, which is incorporated herein by reference, relates to integrating data from a training simulator with video of the training session. The system disclosed in U.S. Patent Application Publication 2008-0145830 is not well adapted for portable applications. See also U.S. Pat. No. 8,113,844 for similar teachings.
There remains a need for the integrated recording and playback of video, audio and multiple user selectable sensor inputs using portable tablet computer and implementing user controlled commenting and flagging.